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Abstract. The compound eyes of mantis shrimps, a group 
of tropical marine crustaceans, incorporate principles of 
serial and parallel processing of visual information that may 
be applicable to artificial imaging systems. Their eyes in- 
clude numerous specializations for analysis of the spectral 
and polarizational properties of light, and include more 
photoreceptor classes for analysis of ultraviolet light, color. 
and polarization than occur in any other known visual 
system. This is possible because receptors in different re- 
gions of the eye are anatomically diverse and incorporate 
unusual structural features, such as spectral filters. not seen 
in other compound eyes. Unlike eyes of most other animals, 
eyes of mantis shrimps must move to acquire some types of 
visual information and to integrate color and polarization 
with spatial vision. Information leaving the retina appears to 
be processed into numerous parallel] data streams leading 
into the central nervous system, greatly reducing the ana- 
lytical requirements at higher levels. Many of these unusual 
features of mantis shrimp vision may inspire new sensor 
designs for machine vision. 


Introduction 


Of all the senses, vision provides animals with the most 
precise spatial registration of the external world and of 
objects in it. Animal vision operates in the spectral range 
between 300 and 750 nm, but technology has extended the 
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“visual” senses of artificial sensors to cover the spectrum 
from X-rays to microwaves. Nevertheless, engineers can 
learn much from the mechanisms that animals use to image 
the world; to map it to self-centered coordinates: to segment 
the physical properties of light transmitted, scattered, or 
reflected by objects: and finally to extract the critical fea- 
tures necessary for survival. Like autonomous vehicles, 
animals must distinguish and recognize items of interest 
(e.g., landmarks, predators, prey, and conspecifics); they 
need to discriminate signals from noise; and they must find 
their way around a geometrically complicated and visually 
confusing world. 

Biological systems are far more constrained than artificial 
ones in their choice of materials and in the means by which 
information is transduced and transmitted. More fundamen- 
tally, living visual systems are products both of evolution 
and development. Evolutionary history acts as a bottleneck, 
greatly limiting the options available in the design of a 
modern animal's eye. Also, the eye must pass through a 
biologically feasible developmental sequence. If the animal 
is one that undergoes metamorphosis (like many insects and 
crustaceans), the developing eye must also be functional for 
the larva’s habitat and biological demands, which may 
affect the eye design available to the adult. 

Here, we consider the highly evolved eyes of stomatopod 
crustaceans, commonly known as mantis shrimp. These 
marine invertebrates possess compound eyes both as adults 
and as larvae. Compound eyes are bulky, and they provide 
very poor spatial resolution for their size. These limits exist 
because each photoreceptor unit of a compound eye, termed 
a rhabdom, is paired with an individual optical system. 
Thus, the compound eye type demands an immense prolif- 
eration of optical units, increasing its size and complexity, 
and ultimately limiting the total number of receptors to a 
few thousand at best (compared to many millions in a 
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vertebrate-type eye). In compound eyes, the unitary struc- 
ture composed of optics, photoreceptors, and associated 
pigment cells is called an ommatidium. 

Compound eyes, nevertheless, do offer certain possibili- 
ties not available to the much more compact camera eye 
design. Normally, all photoreceptors in a single ommiatid- 
ium view an identical receptive field, which may be imaged 
onto several photoreceptor types for spectral or polariza- 
tional analysis. Furthermore, the geometry of the compound 
eye is very flexible, and it is even possible for ommatidia in 
well-separated regions of the eye to view the same spatial 
location. These options provide for considerable flexibility 
in the analysis of an image, and greatly multiply the possi- 
bilities for parallel processing. Indeed, much of the image 
analysis performed in the visual systems of mantis shrimps 
occurs very early in the visual process (sometimes at the 
level of single photoreceptor cells), producing a well-ana- 
lyzed and much-simplified data stream flowing into higher 
visual centers. The compound eye design serves as an 
excellent model for artificial sensors in its modularity and in 
its ability to analyze image features and characteristics 
independently in each unit. 

In this paper, we consider some of the specializations 
found in mantis shrimp compound eyes that permit both 
serial and parallel analysis of visual stimuli. We also de- 
scribe some other unusual features of these eyes that permit 
the decomposition of images into their polarizational, spec- 
tral, spatial, and depth-plane features. Wherever possible. 
we indicate potential applications of mantis shrimp visual 
design to the development of autonomous systems. 


Mantis Shrimp Compound Eyes 


As their common name implies, the stomatopods look 
much like other modern crustaceans. This appearance belies 
their distinctive evolutionary history, which has been sepa- 
rate from the crustacean mainline for about 400 million 
years. Unlike other crustaceans, they actively hunt prey and 
disable or kill it with a catapult-like blow of a specialized 
raptorial appendage. They also can severely injure other 
members of their own species, an ability that has encour- 
aged the evolution of complicated signaling behavior. often 
involving color or polarization features. Among the most 
interesting adaptations are those involving the structure and 
function of their compound eyes. 

Figure ] shows the eyes of Neogonodactylus oerstedii, a 
mantis shrimp species from the Caribbean. Superficially, 
these ovoid eyes appear to be similar to those of other 
crustaceans, or even insects, but a closer inspection reveals 
some important differences. Although most of the eye con- 
sists of a standard hexagonal array of ommiatidia. there is a 
band of six parallel rows of specialized ommatidia running 
around the equator of each eye, looking something like a tire 
tread in the figure. These divide the eye into two roughly 


equal groups of receptors. forming two approximately 
hemispherical halves (see also Manning ef al., 1984). Most 
ommatidia throughout each of these two hemispheres share 
visual fields with ommatidia in the other, providing the 
possibility of stereoscopic vision in a single eye (Marshall 
and Land, 1993). The various ommiatidial rows of the mid- 
band are individually specialized for spectral and polariza- 
tional analysis. We describe these specializations in detail 
later in the paper. 

Another unusual feature is that the eyes are mounted on 
stalks. Stalked compound eyes occur throughout the Crus- 
tacea, but in mantis shrimps the movement of the eye on the 
stalk is unusually free, with the eye being driven in all 
possible axes of movement by six functional groups of 
muscles (Jones, 1994). Thus, whereas eye movements in 
other crustaceans are used to stabilize the eye in the visual 
field, mantis shrimp eyes are impressively and spontane- 
ously active, giving the animals an air of curiosity and 
intelligence. However, the movements of the two eyes seem 
entirely uncoordinated, which is disconcerting to watch. 


Specializations for the analysis of polarized light 


Even though it is not recognized by our visual systems, 
polarized light is abundant in nature, being produced by 
reflection or scattering from almost any object. Analysis of 
partially linearly polarized light. the most common form, 
can be used to infer the surface texture or orientation of an 
object, and many animals use natural polarized-light fields 
for navigation. Machine-vision systems are beginning to 
incorporate polarization-vision concepts for material iden- 
tification, surface-orientation detection, and other applica- 
tions (see Wolff, 1997). The combination of spatial, spec- 
tral, and polarizational analysis in sensors may be simplified 
by resorting to the compound eye design as a model, and the 
analytical synthesis of these independent visual modalities 
may substantially increase the information content of arti- 
ficial images. Technically, the addition of polarization ana- 
lytical ability to artificial imagers is rather simple (Wolff, 
1997). 

The structure of arthropod photoreceptors naturally 
makes them sensitive to the plane of polarization of incom- 
ing light, because the visual pigments that absorb light are 
dichroic and are oriented mostly paralle! in small, tubular 
membranes called microvilli. Most crustaceans take advan- 
tage of this sensitivity by producing ommatidia that contain 
two photoreceptor classes. Each class has microvilli that are 
all parallel but are perpendicular to those of the other 
receptor class, together producing a system with two ana- 
lyzers, each maximally sensitive at an e-vector orientation 
orthogonal to the other (see Waterman, 1981). Mantis 
shrimps are no exception here, and ommatidia throughout 
the eye (except in parts of the midband, as described later) 
are all polarization-sensitive. 
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Figure 1. The anterior end of an adult mantis shrimp, Neogonodactylus oerstedii, imaged by scanning 
electron microscopy. This species is fonnd throughout the Caribbean, living in coral rock and coralline algae. 
The visual regions of the compound eyes, revealed here by the densely packed facets (each of which represents 
the cornea of a single ommatidium), lie at the ends of stalks that pivot at their proximal ends. Each compound 
eye consists of two flattened hemispheres separated by six parallel rows of ommatidia, called the midband. 
Magnification: approx. 30. 


Figure 2 is a diagram of a typical ommatidium from the 
hemispherical regions of the eye outside the midband. The 
top half of this structure, including the refracting cornea and 
the underlying crystalline cone, is devoted to optics. focus- 
ing light on the top of the photosensitive structure, called the 
rhabdom. This rhabdom is constructed from eight receptor 
cells formed into a single light guide. It consists of a small 
upper tier, produced by receptor cell number 8 (R8), and a 
much longer underlying tier formed by contributions from 
receptor cells numbered 1 to 7 (RI-7). R8 is sensitive only 
to ultraviolet light (Cronin et al., 1994b; Marshall and 
Oberwinkler. 1999), but the fused receptor produced by 
R1-7 is sensitive to middle-wavelength light, near 500 nm 
(see Cronin and Marshall. 1989a, b). The R8 receptor is 
polarization-insensitive, as its microvilli are not all parallel, 
but the two sets of receptors in the main rhabdom have 
orthogonal microvilli and are very sensitive to the plane of 
polarization (see also Marshall, 1988: Marshall et al., 
1991a). Axons from these two receptor sets converge onto 
higher order interneurons (Horwood and Marshall. unpubl. 
obs.), which could explain how polarization opponency 
arises in mantis shrimp vision (see Yamaguchi et al., 1976). 


Thus, information leaving the outer regions of the retina has 
already undergone processing that emphasizes differences 
in the signals received by paired sets of receptors. 

Ommatidia like those of Figure 2 are also typical of crabs 
and other “standard” crustaceans. In contrast, the ommatidia 
of the six rows of the midband are unique to the stomato- 
pods. For convenience in discussing the function of mid- 
band ommatidia, we have grouped them into three types 
(Fig. 3). each of which normally appears in two of the six 
ommatidial rows of the midband. These types vary primar- 
ily in the construction of the rhabdom, so only this part of 
each ommatidium is diagrammed in Figure 3. 

Type 1 (found in the two most ventral rows of the mid- 
band) is the simplest and is specialized for the analysis of 
polarized light, like the rhabdoms in ommatidia of the 
retinal hemispheres. Here, however. the R8 (ultraviolet) 
receptor is much enlarged and has microvilli that are all 
parallel and thus polarization-sensitive. The rest of the rhab- 
dom, formed by receptors R1-7, is normal, providing two 
orthogonal receptor channels. Thus, the overall structure has 
a single ultraviolet-sensitive polarization analyzer on top 
and two orthogonal middle-wavelength (near 500 nm) ana- 
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tion. The overlying R8 receptor, with its parallel microvilh, 
may serve as a quarter-wavelength retarder. If so, it would 
convert circularly polarized light to linearly polarized light, 
which could then be analyzed by the polarization-sensitive 
cells of the main part of the rhabdom. Circular polarization 
may be produced naturally by reflection from metals or 
certain birefringent structures (such as crusiacean cuticle), 
or by transmission through birefringent material or solutions 
of some complex molecules. The ability to detect and ana- 
lyze circular polarization would add yet another information 
channel to the data stream flowing into the central nervous 
system. Note also that the geometrical arrangement of the 
receptors (i.e., their segregation into tiers in a single rhab- 
dom) permits each level both to operate on its own and also 
to affect the operation of receptors placed more deeply in 
the overall retina. 
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Figure 2. Diagram of a typical ommatidium from the hemispherical 
region of a mantis shrimp compound eye. Proceeding from surface of the 
ommatidium (al the top) inwards, it consists of a cornea, a crystalline cone Y 
(the tong, tapered structure), the eighth receptor cell (R8), and a group of es R1-7 Uy R8 
receptor cells numbered 1 through 7. The receptors form a rhabdom, which 
is diagonally hatched in the RS region and crosshatched in the R1-7 region, SS S; 
and each receptor cell terminates in an axon that exits the ommatidium al A R1,4,5 UW; R2,3,6,7 
the base. The receptor region is shielded from off-axis light by an overlying 


layer of proximal pigment. 
E rers 


lyzers in the main part of the rhabdom. A second ultraviolet Figure 3. Diagrams of the rhabdomal portions of ommatidia located in 
system, perpendicular to the first, is available as well, be- the midband regions of compound eyes of most stomalopod species. Each 


cause the Iwo ommatidial rows with the Type l ommatidium rhabdom has an R8 receptor cell, which has its sensitivity maximum in the 
uitraviolet, on top. The main rhabdom (R 1-7) exists in a single tier in Type 


are rotated 90° to each other. Together, these systems pro- A | eee i 
ide: i] ae . TEN i t ommatidia (left), but is divided into two tiers by separating Rie, and 5 
vide excellent polarization vision in two separate spectra from R2, 3, 6, and 7 in Types tf (center) and II (right). Type III also has 


regions. colored, photostable filters at the junctions between tiers. See the text for 


The Type I ommatidium has an addilional potential func- further details. 


VISION IN MANTIS SHRIMPS E 


Specializations for the spectral analysis of light 


The spectrum of light reflected from an object gives rise 
to the psychophysical perception of color. Color is an in- 
herent property of an object, useful for object recognition 
and identification. It also can be used to discriminate objects 
that have similar brightnesses but different reflectance spec- 
tra and to recognize borders. Many animals, including man- 
tis shrimps, use colored markings to produce highly recog- 
nizable and unambiguous signals. 

The two themes just discussed with reference to polar- 
ization vision—tiering of receptors to modify the responses 
of receptors at deeper levels and immediate processing of 
receptor output into opponent channels— continue with fur- 
ther elaborations in the spectral analysis systems of mantis 
shrimp compound eyes. These retinal specializations give 
mantis shrimps the potentially most complicated color vi- 
sion systems of any animals. Ommatidia specialized for 
spectral analysis are those of Types II and III (Fig. 3: see 
also Marshall, 1988: Marshall et al., 199]a, b). 

Type I] ommatidia are Jocated in two of the four most 
dorsal ommiatidial rows of the midband. In these, the main 
trhabdom that formed a single tier in ommatidia specialized 
for polarizational analysis is divided into two tiers. There- 
fore, light entering the rhabdom transits first the ultraviolet- 
sensitive R8 receptor, then the distal tier of the main rhab- 
dom, and finally the main rhabdom’s proximal tier. 

Visual pigments, the molecules that absorb light and 
trigger the biochemical processes of vision, modify incom- 
ing light because they absorb within particular spectral 
regions. Rhabdoms of Type I] ommatidia thus operate as a 
series of spectral filters, each of which affects all receptors 
beneath. Unlike the situation in other ommatidia, in Type H 
main rhabdoms (as well as those of Type NI, to be described 
shortly), the distal tier and proximal tiers of main rhabdoms 
contain different visual pigments. In every case, the pigment 
of the distal tier absorbs at shorter wavelengths than that of 
the proximal tier, thereby acting as a long-pass filter. Each 
tier of Type I] rhabdoms sharpens and tunes the spectral 
sensitivity of the receptors below, producing a set of spec- 
trally specialized, narrow-band photoreceptors (see Cronin 
and Marshall, 1989a, b). 

The serial filtering ot Type IT ommatidia ts further elab- 
orated in Type HI (Fig. 3), found in the remaining two 
ommiatidial rows of the midbands of most mantis shrimp 
species. Type I rhabdoms intercalate colored filters, made 
of tightly packed vesicles containing strongly absorbing. 
photostable pigments, at each successive junction between 
tiers (see Marshall, 1988; Cronin and Marshall. 1989a, b; 
Marshall et al., 199la, b; Cronin et al. 1994a). Light 
entering each tier is therefore altered not only by absorption 
by visual pigments at all higher levels, but also by the 
colored filters that it has transited. Type HI rhabdoms are 
particularly suitable for producing middle-to-long-wave- 


length receptor classes. Visual pigments tend to absorb 
poorly at these wavelengths (thus making weak filters), but 
the photostable filters are very effective for such tuning. 

The tiers of the main rhabdoms in Type II and Type III 
ommatidia are formed from the same receptor sets that 
produce the orthogonal polarization channels in the remain- 
der of the retina. Thus, of the seven receptor cells that 
contribute to the main rhabdom. those numbered 1. 4, and 5 
produce one polarization channel. while 2. 3, 6, and 7 
produce the other. These same groupings are retained tn the 
tiers of Type I] and Type III ommiatidia (see Fig. 3), and this 
converts what was originally a polarization-opponent orga- 
nization into chromatic-opponent channels (see Cronin and 
Marshall, 1989a, b; Marshall et al., 199la, b, 1996). To 
prevent confusion between polarization and spectral com- 
ponents, receptors in Type lI and Type I rhabdoms are 
polarization-insensitive, having randomly oriented micro- 
villi (Marshall er al., 1991b). 

Since four rows of the midband include ommatidial 
Types II and HI, eight varieties of receptor tiers exist in their 
main rhabdoms. Each of these contains a different visual 
pigment (Cronin and Marshall, 1989a, b), producing eight 
distinct classes of spectral receptor. Together, these sharply 
tuned receptors span the visual spectrum to peak at wave- 
lengths from about 400 nm to nearly 700 nm in some 
species. To these are added a diversity of ultraviolet recep- 
tor types (in the RS cells) and also the polarization classes, 
for a grand total of perhaps 16 spectral receptor classes (see 
Cronin et al., 1994c; Marshall and Oberwinkler, 1999). 

With all these receptor types, one might conclude that no 
crustacean brain could possibly interpret such a complex, 
multivariate set of incoming data. Obviously, the polariza- 
tion system would make things even worse. However, the 
actual situation is probably quite the opposite. By handling 
incoming information with a large number of individually 
specialized receptors (groups of which analyze the same 
location in visual space) and by processing receptor outputs 
immediately. sensory information leaving the retina is al- 
ready streamed into a parallel series of data channels. Each 
of these may act as a single labeled line of visual informa- 
tion. 

The division of the visual spectrum into a series of 
discrete channels has another, less obvious, advantage. It is 
difficult for most visual systems (and systems of artificial 
imaging) to achieve color constancy—the ability to recog- 
nize a given color unambiguously— underwater or in spec- 
trally challenging environments. Different classes of color 
receptors normally respond to a broad spectrum of light, and 
this can cause them to adapt strongly to stimuli that are 
spectrally far from their wavelengths of peak sensitivity. By 
having narrow, sharply tuned spectral classes, a visual sys- 
tem reduces this adaptation to off-peak wavelengths, and 
thereby maintains excellent color constancy (Osorio et al., 
1997). Mantis shrimps use color flamboyantly in their sig- 
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nals (Caldwell and Dingle. 1975). more so than any other 
aquatic invertebrate. Their unique system of color vision 
(see also Marshall er al.. 1996: Chiao er al.. 2000) makes 
this possible. When color constancy is cnucal. artificial 
designs should use narrow, well-chosen spectral channels 
for accuracy. 


Spatial and motion analysis. and the integration of visual 
information 


Being active predators that incapacitate their prey using a 
ballistic sinke. mantis shnmps require a very high-quality 
sense of space. Again. their eves are uniquely specialized 
for this purpose as well. The overlapping fields of view 
shared by the two halves of each eve probably form the 
basis of a monocular rangefinder. whereby the distance to a 
particular object determines which particular receptor pair 
converges onto it. Since the stalked eves of mantis shrimps 
(Fig. 1) move freely in pitch. roll. and vaw. binocular 
stereopsis is difficult or impossible for these animals to 
achieve. So monocular rangefinding is particularly suitable 
(Milne and Milne. 1961). 

The very mobility of the eves of manus shrimps can 
confound their interpretation both of spatial location and of 
relative motion. Like most animals with mobile eves. the 
manus shrimps solve this problem by immobilizing their 
eves most of the time relative to the overall! visual field. and 
making rapid eve movements (saccades) to object of interest 
(Cronin er al.. 1991). They mav also track individual objects 
moving within the visual background (Cronin er al., 1988S. 
1992). These stabilizing mechanisms. necessary to anv vi- 
sual system. whether biological or artificial. permit mantis 
shrimps to recognize self-motion or mouon of other objects 
in their vicinity. 

Mantis shnimps move their eves for far more unusual 
purposes. In many species. including Veogonodacnlus oer- 
stedii, the ommatidia of the compound eve are located at the 
end of a relatively long stalk. the pivot of which is far from 
the visual region of the eve (see Fig. 1). Consequently. as 
the eve swings. its motion prodnces a visual flow field in 
which more distant objects move slowly relative to nearby 
objects. This relanve motion provides an additional depth 
cue, Operating at substantially greater range than the mon- 
ocular rangetinders. that could be used to distinguish fore- 
ground trom background. The basic design principle is that 
the nodal point of the optics ts removed trom the geomet- 
rical center of rotation: some vertebrates (e.¢.. the sandlance 
fish and chameleons) have hit on a similar solution to the 
problem of monocular rangefinding (Ou and Schaeffel. 
1995: Pettigrew er al.. 1999). 

Equally unusual are the small scanning movements that 
mantis shnmps use. apparently to map their spatially re- 
stricted senses of color and polarization onto the extended 
visual fields of the hemispheric regions of the compound 


eve. The problem that these animals must solve is a conse- 
quence of the geometry of the midband. Since its six rows 
of receptors all lie in a single plane. together they sample 
only a planar slice through visual space. Yet this slice 
contains the most highly analyzed chromatic. ultraviolet. 
and polarization information. To “paint” color. ultraviolet 
intensity. and polanzation onto the extended visual field. 
manus shnmps use small and slow scanning movements. 
These sweep the planar visual fields of midband ommatidia 
over the broad and extended visual fields of ommatidia in 
the hemispherical regions of the compound eve (Land er al., 
1990). Engineers often attempt to minimize sensor motion 
relative to the external world. but in mantis shnmps. motion 
is essential for visual function. As is also implied in the 
work of Srinivasan and his colleagues (2001). sensor mo- 
tion can add information. and instrument design incorporat- 
ing sensor movement deserves further study. 


Summary and Conclusions 


Mantis shnmps are among the most successful inverte- 
brate predators in shallow. tropical waters. They are formi- 
dable hunters. either cruising about the bottom actively in 
search of prey or stealthily ambushing it from a concealed 
burrow. Their success is largely due to their extraordinary 
visual senses. which perform much of the required lower- 
level processing of incoming visual information before re- 
laving it to higher visual centers. In doing so. they consid- 
erably reduce the computational overhead required in the 
central nervous system. permitting decisions to be made 
rapidly and accurately. Although the central nervous system 
of stomatopods is poorly studied at present. it seems likely 
that the extensive retinal pre-processing may guide appro- 
priate motor output with relatively minor amounts of central 
processing. These principles should find excellent applica- 
uon in artificial autonomous constructs as well. and can 
serve as models for the design of systems of artificial vision. 
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